Image sensing system and its control method

ABSTRACT

In an image sensing system having an image sensing optical system, and an image sensing element for photoelectrically converting incoming light from the image sensing optical system, a predetermined pattern image for adjustment, which is specified in advance, is read by the image sensing element, and the image sensing element is driven to adjust its position on the basis of an output from said image sensing element, thereby adjusting the relative position of the image sensing element with respect to the image sensing optical system prior to an image sensing operation.

FIELD OF THE INVENTION

The present invention relates to an electrical image sensing systemusing an image sensing optical system and an image sensing element suchas a photoelectric conversion element and its control method and, moreparticularly, to an image sensing system that can adjust the relativestate between the image sensing optical system and image sensing elementand its control method.

BACKGROUND OF THE INVENTION

FIG. 23 shows a typical arrangement of a conventional image sensingsystem, and the operation of the conventional image sensing system willbe briefly explained below with reference to FIG. 23. In this imagesensing system, an object image is formed on an image sensing element 1by an image sensing optical system 8, which has a focal lengthadjustment optical system L1 for adjusting the image sensing fieldangle, a correction optical system L2 for making correctioncorresponding to the movement of the optical system L1, a shift opticalsystem L3 for camera-shake correction, an iris mechanism (Iris) foradjusting the amount of incoming light, and a focal point positionadjustment optical system L4 for making focus adjustment. The imagesensing element 1 photoelectrically converts the object image into anelectrical signal, and a signal processor 3 processes the electricalsignal to obtain a color video signal. The color video signal is output,and its luminance information is mainly supplied to an exposure control(AE) unit 4 and focus adjustment (AF) unit 2, thus generating respectivecontrol signals (see Japanese Patent Laid-Open Nos. 3-159377 and8-190113).

The AE unit 4 controls the accumulation time (so-called shutter speed)of the image sensing element 1 for each frame, and the iris mechanismIris, and the AF unit 2 controls the focal point position adjustmentoptical system L4.

A vibration sensor 7 comprises an acceleration sensor and the like, anddetects any camera shake. A shift optical system driving (AS/IS) unit 6reduces blur of an object image caused by camera shake by driving theshift optical system L3 using the detection output of the vibrationsensor 7.

A field angle adjustment (zoom) unit 5 receives an operation instructionsignal for adjusting the image sensing field angle in accordance with auser's request, reads out a digital cam curve from a memory (not shown),and controls the focal length adjustment optical system L1, correctionoptical system L2, and focal point position adjustment optical system L4in cooperation with each other.

As described above, in recent years, along with a rapid size reductionof the image sensing system, a multi-function, high-performance, compactimage sensing system can be realized. For example, Japanese PatentLaid-Open No. 3-159377 discloses contents associated with an electrictilt & shift control mechanism and multi-point AF evaluation. However,the image sensing element requires very high attachment precision.

For example, as shown in FIG. 24, an angle θ the optical axis of theimage sensing optical system 8 makes with the image sensing element 1often tilts from the vertical or horizontal position due to poormanufacturing precision of parts, attachment errors in the manufacturingprocess, and the like. In such case, as the system size becomes smaller,it becomes harder to make the angle θ fall within an allowable range,and it becomes difficult to adjust the angle at high precision thatmatches the current size reduction requirement.

Furthermore, as the image sensing element is down-sized, the focallength of the image sensing optical system becomes shorter, and thedepth of field tends to become deeper. As a result, it becomes difficultto use a photographing technique that controls the depth of field (e.g.,a shallow depth of field is set to make the background out of focus).

SUMMARY OF THE INVENTION

The present invention has been made in consideration of the abovesituation, and has as its object to provide an image sensing systemwhich can easily and accurately attain automatic adjustment of therelative position (angle) of an image sensing element within apredetermined allowable range with respect to an image sensing opticalsystem by setting of an operation instruction (parameter) by theoperator, and can sufficiently cope with a further size reduction of theapparatus arrangement, and its control method.

It is another object of the present invention to suppress deteriorationof image quality due to tilt of an image sensing element in an imagesensing apparatus having a field angle adjustment function.

According to the present invention, the foregoing object is attained byproviding an image sensing system comprising: an image sensing elementfor photoelectrically converting incoming light from an image sensingoptical system; and an adjuster for adjusting a position of the imagesensing element as an initial operation for an image sensing operation,wherein the adjuster adjusts a relative position of the image sensingelement with respect to the image sensing optical system by reading anadjustment pattern image using the image sensing element, and driving toadjust the position of the image sensing element on the basis of anoutput from the image sensing element.

According to the present invention, the foregoing object is alsoattained by providing an image sensing system comprising: an imagesensing element for photoelectrically converting incoming light from animage sensing optical system; an optical member which is inserted beforethe image sensing element to independently adjust optical path lengthsin horizontal and vertical directions; and an adjuster for adjusting anoptical path of an optical image that enters the image sensing elementby controlling the optical member prior to an image sensing operation,wherein the adjuster reads an adjustment pattern image using the imagesensing element, and adjusts the optical member to optimize a highfrequency component of a signal output from the image sensing element.

Further, the foregoing object is also attained by providing a method ofcontrolling an image sensing system which comprises an image sensingelement for processing incoming light from an image sensing opticalsystem, comprising: an input step of reading an adjustment pattern imageusing the image sensing element as an initial operation for an imagesensing operation; and a position adjustment step of adjusting arelative position of the image sensing element with respect to the imagesensing optical system by driving to adjust a position of the imagesensing element on the basis of an output from the image sensingelement.

Furthermore, the foregoing object is also attained by providing a methodof controlling an image sensing system which comprises an image sensingelement for processing incoming light from an image sensing opticalsystem, and an optical member which is inserted before the image sensingelement to independently adjust optical path lengths in horizontal andvertical directions, comprising: an input step of reading an adjustmentpattern image using the image sensing element prior to an image sensingoperation; and an adjustment step of the optical member to optimize ahigh frequency component of a signal output from the image sensingelement.

Further, the foregoing object is also attained by providing an imagesensing system comprising: an image sensing element forphotoelectrically converting incoming light from an image sensingoptical system; a memory for storing information that pertains to arelative position between the image sensing element and the imagesensing optical system; and an adjuster for adjusting a position of theimage sensing element, wherein the adjuster drives based on theinformation read out from the memory to adjust the relative position ofthe image sensing element with respect to the image sensing opticalsystem.

Further, the foregoing object is also attained by providing a method ofcontrolling an image sensing system which comprises an image sensingelement for photoelectrically converting incoming light from an imagesensing optical system, wherein information that pertains to a relativeposition of the image sensing element with respect to the image sensingoptical system is stored, and a position adjuster is controlled toadjust the relative position of the image sensing element with respectto the image sensing optical system on the basis of the information.

Further, the foregoing object is also attained by providing an imagesensing system comprising: an image sensing element forphotoelectrically converting incoming light from an image sensingoptical system; and an adjuster for adjusting a relative state betweenthe image sensing optical system and the image sensing element on thebasis of an operator's operation instruction.

Further, the foregoing object is also attained by providing an imagesensing optical system which is detachably attached to an image sensingapparatus having an image sensing element, comprising: a plurality oflenses; and a memory that stores an adjustment amount used to correct atilt of an optical axis of the image sensing optical system with respectto a reference plane to have a predetermined relative state.

Further, the foregoing object is also attained by providing an imagesensing apparatus to which an image sensing optical system is detachablyattached, comprising: an image sensing element for photoelectricallyconverting incoming light from the image sensing optical system; and anadjuster for adjusting a relative state between the image sensingoptical system and the image sensing element on the basis of anoperator's operation instruction.

Further, the foregoing object is also attained by providing a method ofcontrolling an image sensing system having an image sensing element forphotoelectrically converting incoming light from an image sensingoptical system, wherein a relative state between the image sensingoptical system and the image sensing element is adjusted on the basis ofan operator's operation instruction.

Other features and advantages of the present invention will be apparentfrom the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention and,together with the description, serve to explain the principles of theinvention.

FIG. 1 is a block diagram showing the overall arrangement of an imagesensing system according to the first embodiment of the presentinvention;

FIG. 2 shows the basic concept of angle adjustment;

FIG. 3 shows the concept of intra-frame integrated value adjustment intwo directions of X- and Y-axes;

FIG. 4 is a flow chart showing the initial adjustment sequence accordingto the first embodiment of the present invention;

FIG. 5 is a block diagram showing the overall arrangement of an imagesensing system according to the second embodiment of the presentinvention;

FIG. 6 is a flow chart showing the initial adjustment sequence accordingto the second embodiment of the present invention;

FIG. 7 is a block diagram showing the overall arrangement of an imagesensing system according to the third embodiment of the presentinvention;

FIGS. 8A and 8B show the relative relationship between an image sensingoptical system and image sensing element;

FIG. 9 is a schematic diagram showing a method of measuring and storingan initial position using a reference image sensing element;

FIG. 10 is a flow chart showing the initial adjustment sequenceaccording to the third embodiment of the present invention;

FIG. 11 is a block diagram showing the overall arrangement of an imagesensing system according to the fourth embodiment of the presentinvention;

FIG. 12 shows an error table in the image sensing system according tothe fourth embodiment of the present invention;

FIG. 13 is a flow chart showing the initial adjustment sequenceaccording to the fourth embodiment of the present invention;

FIG. 14 is a block diagram showing the overall arrangement of an imagesensing system according to the fifth embodiment of the presentinvention;

FIG. 15 is a flow chart showing the initial adjustment sequenceaccording to the fifth embodiment of the present invention;

FIG. 16 is a block diagram showing the overall arrangement of an imagesensing system according to the sixth embodiment of the presentinvention;

FIG. 17 is a flow chart showing the initial adjustment sequence in thesixth embodiment of the present invention;

FIG. 18 is a block diagram showing the overall arrangement of an imagesensing system according to the seventh embodiment of the presentinvention;

FIG. 19 is a flow chart showing the position adjustment operation upontaking a photo with the photographic effect according to the seventhembodiment of the present invention;

FIGS. 20A to 20C show a method of setting in-focus and non-in-focusregions according to the seventh embodiment;

FIGS. 21A to 21C show another method of setting in-focus andnon-in-focus regions;

FIG. 22 is a schematic view for explaining an inverse tilt and shifteffect according to the seventh embodiment;

FIG. 23 is a block diagram showing the overall arrangement of aconventional image sensing system; and

FIG. 24 is a schematic view showing a variation of an angle that theoptical axis of an image sensing optical system makes with an imagesensing element.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Preferred embodiments of the present invention will be described indetail in accordance with the accompanying drawings.

<First Embodiment>

FIG. 1 is a block diagram showing the overall arrangement of an imagesensing system according to the first embodiment of the presentinvention.

In this image sensing system, reference numeral 1 denotes an imagesensing element which comprises a photoelectric conversion element suchas a CCD, CMOS, or the like; and 8, an image sensing optical systemwhich has a focal length adjustment optical system L1 for adjusting theimage sensing field angle, a correction optical system L2 for makingcorrection corresponding to the movement of the optical system L1, ashift optical system L3 for camera-shake correction, an iris mechanism(Iris) for adjusting the amount of incoming light, and a focal pointposition adjustment optical system L4 for making focus adjustment.

Furthermore, reference numeral 2 denotes a focus adjustment (AF) unitfor controlling the focal point position adjustment optical system L4;3, a signal processor for receiving a sensed image signal from the imagesensing element 1, and outputting an image; 4, an exposure control (AE)unit for controlling the accumulation time (so-called shutter speed) ofthe image sensing element 1 for each frame, and the iris mechanism Iris;and 5, a field angle adjustment (zoom) unit, which receives an operationinstruction signal for adjusting the image sensing field angle inaccordance with a user's request, reads out a digital cam curve from amemory (not shown), and controls the focal length adjustment opticalsystem L1, correction optical system L2, and focal point positionadjustment optical system L4 in cooperation with each other.

Reference numeral 7 denotes a vibration sensor which comprises anacceleration sensor and the like, and detects any camera shake; and 6, ashift optical system driving (AS/IS) unit for reducing blur in an objectimage caused by camera shake by driving the shift optical system L3 uponreceiving the signal from the vibration sensor 7.

Reference numerals 12 and 11 denote drivers which respectively pivot theimage sensing element 1 about the X- and Y-axes, and can use, e.g., astepping motor, voice coil motor, or an actuator such as a piezoelectricelement or the like.

Reference numeral 13 denotes a panel formed with a test chart; 10, apanel driver for moving the panel 13; 14, an illumination unit forilluminating the test chart on the panel 13; and 15, an emissioncontroller for controlling to drive the illumination unit 14.

The test chart includes various frequency components ranging from lowfrequency components with a large pattern spacing to high frequencycomponents with a small pattern spacing. The illumination unit 14 canselect a plurality of wavelengths ranging from a reddish color having along wavelength to a bluish color having a short wavelength. A whitelight source may be used.

A system controller 9 controls the AF unit 2, panel driver 10, andemission controller 15.

A characteristic feature of the image sensing system according to thefirst embodiment of the present invention lies in that the relativeposition (angle) of the image sensing element 1 with respect to theoptical system 8 is initially adjusted prior to steady control of theimage sensing optical system 8 and its various parameters on the basisof control data from the respective processors (AF unit 2, AE unit 4,AS/IS 6, zoom unit 5, and Iris).

FIG. 2 shows the basic concept of an image sensing region when the imagesensing element 1 undergoes angle adjustment.

The angle adjustment must be done for the image sensing frame about theX- and Y-axes. An inscribed maximum rectangular image sensing area in aneffective image circle that includes an effective image formed by theimage sensing optical system 8 is area A, and an eclipse-free safe areathat takes the angle adjustment of the image sensing element 1 about thetwo axes is area B. In this case, an image signal is generated andoutput using as area B an effective image sensing area obtained byadjusting an angle θ (θx, θy) the optical axis of the image sensingoptical system 8 makes with the image sensing element 1 with respect tothe X- and Y-axes.

A method of determining and storing the initial state of the imagesensing element 1 will be described below.

In order to measure any tilt of the image sensing element 1, the panel13 with the measurement test chart having frequency components in the X-and Y-axis directions is read. The panel 13 is located in front of theimage sensing optical system 8, and can also serve as a lens protectionbarrier depending on the design. When the test pattern is set at a verynear distance position in this way, the system controller 9 issues amovement instruction toward the wide-angle end to the zoom unit 5 toallow photographing at the closest focal length position.

The emission controller 15 controls the illumination unit 14 to emitlight in synchronism with the panel driving timing of the panel driver10. The test pattern reflects that illumination light, and reflectedlight forms an image on the image sensing element 1 via the imagesensing optical system 8. Image data obtained by the image sensingelement 1 undergoes a predetermined signal process by the signalprocessor 3, thus outputting a color video signal.

The frequency components of the test pattern obtained from the signalprocessor 3 are analyzed by the AF and AE units 2 and 3 as processors onthe output side, thus making correction and adjustment.

The high frequency components of luminance signal information are mainlyused by the AF unit 2, which controls the drivers 11 and 12 inaccordance with the high frequency component values associated with theX- and Y-axes, thereby adjusting to maximize the intra-frame integratedvalue of the high frequency components.

FIG. 3 shows the concept of intra-frame integrated value adjustment inthe two directions of the X- and Y-axes.

In this case, so-called hill-climbing control using a TV signal is donefor the X- and Y-axes to optimize the whole frame.

For example, the Y-axis value is fixed at an arbitrary value, andhill-climbing control is done so that the high frequency component inthe X-axis direction assumes a highest value. Next, while the X-axisvalue is fixed at the apex of the X-axis, hill-climbing control of aY-axis component is executed, and tilted angles (θx, θy) correspondingto the apex values on the X- and Y-axes are stored.

Furthermore, the aforementioned adjustment may be executed under apredetermined condition or periodically so as to maintain relativeprecision above the apex values (X1, Y1).

The aforementioned adjustment sequence will be explained below withreference to the flow chart shown in FIG. 4.

If the panel driver 10 detects an adjustment start instruction(condition) from the system controller 9 (step S1), the zoom unit 5instructs the focal length adjustment optical system L1 to move towardthe wide-angle end to allow photographing at the closest focal lengthposition (step S2). A lens cover is closed (step S3), the emissioncontroller 15 drives the illumination unit 14 to illuminate the testchart on the panel 13, and the image sensing element 1 senses an imageof the test chart (step S4). Subsequently, the hill-climbing control foradjusting a tilt of the image sensing element 1 about the X-axis isexecuted (step S5), and the hill-climbing control for adjusting a tiltof the image sensing element 1 about the Y-axis is executed (step S6).Upon completion of adjustment about the two axes (YES in step S7), thelens cover is opened (step S8), and the illumination unit 14 is turnedoff at the same time (step S9).

In this way, initial adjustment is completed, and a steady image sensingprocess can be started.

As described above, according to the image sensing system of the firstembodiment, automatic adjustment of the relative tilt between the imagesensing element 1 and image sensing optical system 8 can be easily andaccurately made to fall with a predetermined allowable range by a simplearrangement, and a further size reduction of the apparatus arrangementcan be sufficiently coped with.

Since the test chart is provided in front of the lens of the imagesensing optical system, it can be also used as a lens barrier, and fullautomatic initial adjustment can be implemented.

<Second Embodiment>

The second embodiment of the present invention will be described below.

An image sensing system of the second embodiment initially adjusts therelative tilt between the image sensing optical system 8 and imagesensing element 1 as in the first embodiment, and the first embodimentis applied to an exchangeable lens type image sensing system in thesecond embodiment. Note that the same reference numerals denote the samebuilding components and the like as those in the first embodiment, and adetailed description thereof will be omitted.

FIG. 5 is a block diagram showing the overall arrangement of the imagesensing system according to the second embodiment of the presentinvention.

This image sensing system is roughly divided into a lens unit 100 andcamera unit 200, which respectively comprise microcomputers serving assystem controllers 16 and 18, and execute data communications betweenthese microcomputers.

When the lens unit 100 is connected to the camera unit 200, the imagesensing optical system 8 forms an object image on the image sensingelement 1, which photoelectrically converts the object image. An imagesignal obtained by photoelectric conversion is processed by the signalprocessor 3 to obtain a color video signal. The processed video signalis output, and mainly its luminance information is supplied to the AEand AF units 4 and 2, which generate respective control signals andsupply them to the system controller 18. Control data from the AE and AFunits 4 and 2 are transferred from the system controller 18 to thesystem controller 16 of the lens unit 100 to control the Iris and thefocal point position adjustment optical system L4. The AE unit 4 alsocontrols the accumulation time of the image sensing element 1 for eachframe as needed.

The vibration sensor 7 such as an acceleration sensor or the like in thelens unit 100 detects camera shake, and the shift optical system drivingunit (AS/IS) 6 drives the optical system L3 to reduce the blur of anoptical image caused by the camera shake.

Upon receiving an operation instruction signal for image sensing fieldangle (zoom) adjustment in accordance with an operator's request, thesystem controller 18 transfers it to the system controller 16 in thelens unit 100. The system controller 16 reads out a digital cam curvefrom the internal memory (not shown) of the lens unit 100, and controlsthe optical systems L1 and L2 in cooperation with each other.

The initial adjustment sequence in the second embodiment will bedescribed below with reference to the adjustment flow chart shown inFIG. 6.

FIG. 6 shows an example of the data communication sequence between thesystem controller 16 in the lens unit 100 and the system controller 18in the camera unit 200.

If attachment of the lens is detected (YES in step S11), the systemcontroller 18 sends command data that instructs start of the initialprocess (step S12). Upon receiving this command data, the lens unit 100sets the field angle at the wide-angle end (step S13), controls thepanel driver 10 to insert the test chart of the panel 13 into theoptical path (step S14), drives the illumination unit 14 to illuminatethe test chart, and controls the image sensing element 1 to sense animage of the test chart (step S15). The hill-climbing control foradjusting the tilt of the image sensing element 1 about the X-axis isexecuted by the method mentioned using FIG. 3 (step S16), and thehill-climbing control for adjusting the position of the image sensingelement 1 about the Y-axis is executed (step S17). If adjustment is notcomplete yet (NO in step S18), the flow returns to step S16; otherwise(YES in step S18), an initial process end instruction is sent to thesystem controller 16 of the lens unit 100 (step S19). The lens unit 100retracts the test chart (step S20), and turns off the illumination unit15 for the test chart (step S21), thus allowing normal photographing.

As described above, according to the image sensing system of the secondembodiment, in addition to the effects of the first embodiment, even inthe exchangeable lens type image sensing system, since data associatedwith adjustment can be exchanged via communications between the systemcontroller 18 of the camera unit 200 and the system controller 16 of thelens unit 100, the relative state can be optimized even when imagesensing optical systems having different characteristics are attached tothe camera unit 200. Hence, an inappropriate image sensing frame onwhich an out-of-focus portion locally remains can be excluded, and ahigh-quality image can be sensed.

<Third Embodiment>

The third embodiment of the present invention will be described below.

FIG. 7 is a block diagram showing the arrangement of an image sensingsystem according to the third embodiment of the present invention. Thesame reference numerals in FIG. 7 denote the same parts as in FIG. 1,and a description thereof will be omitted.

The image sensing system in the third embodiment shown in FIG. 7 doesnot have any arrangement associated with the test chart, and furthercomprises an error memory 42 for storing information that pertains tothe relative angle of the image sensing element 1 with respect to theimage sensing optical system 8, and a temperature sensor 61 formeasuring the temperature inside the apparatus.

The information stored in the error memory 42 includes, for example,error information which corresponds to the positional difference betweena reference image sensing element and a reference value obtained basedon a test chart for position adjustment, which has prescribed horizontaland vertical frequency components, for each predetermined temperatureassociated with the relative position of the image sensing element 1with respect to the image sensing optical system 8.

The drivers 11 and 12 adjust the relative state of the image sensingelement 1 with respect to the image sensing optical system 8 to fallwithin a predetermined allowable range on the basis of the informationin the error memory 42.

The system controller 9 controls the drivers 11 and 12 on the basis ofinformation from the error memory 42 and temperature sensor 61.

A characteristic feature of the image sensing system of the thirdembodiment lies in that the relative state of the image sensing element1 with respect to the image sensing optical system 8 is adjusted to fallwithin a predetermined allowable range (angle correction) by driving thedrivers 11 and 12 on the basis of information read out from the errormemory 42 prior to the image sensing operation.

A method of determining and storing the initial state of the imagesensing element 1 will be described below with reference to FIGS. 8A,8B, and 9.

FIG. 8A shows the image sensing optical system 8 which is attached tohave a tilt of θ° with respect to the image sensing element 1. The tiltof the image sensing optical system 8 can be corrected (its optical axiscan be perpendicular to the image sensing element 1) by tilting θ° theimage sensing element 1, as shown in FIG. 8B.

Therefore, in order to measure the tilt of the optical axis of the imagesensing optical system 8 with the aforementioned arrangement, the imagesensing optical system 8 forms an image of a measurement test chart 43having X- and Y-frequency components on a reference image sensingelement 40 set at a reference attachment position. The image sensingelement 40 photoelectrically converts an optical image of the test chart43, and outputs an electrical signal. The photoelectrically convertedsignal is converted by a signal processor 44 into a color (ormonochrome) video signal, which is supplied to an evaluation unit 41.The evaluation unit 41 evaluates the relative position between the imagesensing optical system 8 and reference image sensing element 40. Thereare some evaluation methods, and three methods will be explained below.

(1) The reference image sensing element 40 serving as a reference istilted about the X- and Y-axes to detect a tilt angle corresponding tothe maximum high frequency component.

(2) At an arbitrary measurement position on the reference image sensingelement 40, a focus lens group is moved to maximize the high frequencycomponents, and that position is stored. Similar operation is repeatedat a plurality of different measurement positions on the reference imagesensing element 40, and an optimal tilt angle of the reference imagesensing element 40 is computed from the focus lens group positioninformation for the respective measurement positions.

(3) At an arbitrary measurement position on the reference image sensingelement 40, the test chart 43 is moved to maximize the high frequencycomponents, and that position is stored. Similar operation is repeatedat a plurality of different measurement positions on the reference imagesensing element 40, and an optimal tilt angle of the reference imagesensing element 40 is computed from the position information of the testchart 43 for the respective measurement positions.

The method of computing the optimal tilt angle can use a method based onso-called hill-climbing control that has been explained in the firstembodiment using FIG. 3.

An angle (an error from a reference value) calculated by one of thesethree methods is stored in the error memory 42. This error memory 42comprises a nonvolatile memory such as an EEPROM or the like, and isprovided to a lens unit including the image sensing optical system 8.When this lens unit is attached to a camera main body to assemble theimage sensing system, a system microcomputer in the camera main body canread out the angle stored in the error memory 42.

Conversely, a reference image sensing optical system serving as areference may be prepared, and a tilt angle that the optical axis ofthis reference image sensing optical system makes with the image sensingelement 1 can be acquired by the same operation as described above. Inthis case, a memory for storing this correction angle is provided to thecamera main body having the image sensing element 1. When thiscorrection angle is combined with the angle stored in the error memory42 to obtain an initial value, the relative position between the imagesensing element 1 and image sensing optical system 8 can be moreprecisely adjusted.

In order to further improve the precision, variations of a tilt angledue to a change in temperature in the image sensing system must becorrected. For this purpose, a plurality of pieces of error informationcorresponding to a plurality of temperatures are acquired, and arestored in the error memory 42. For example, tables may be prepared forrespective temperatures, and may store respective parameter values. Fiveor six measurement temperatures in 20° C. increments can be stored tohave ordinary temperature of 20° C. as the center. For example,temperatures are set like −20° C., 0° C., 20° C., 40° C., and 60° C.

In this case, the temperature inside the image sensing system ismeasured by the temperature sensor 61, error information is read outfrom a table with a value closest to the measured temperature of variousparameter values for a plurality of temperatures, which are pre-storedin the error memory 42, and the relative position is corrected inaccordance with the readout value. As a correction method, the drivers11 and 12 adjust the attachment tilt angles of the image sensing element1 about the X- and Y-axes, thus adjusting the relative state between theimage sensing optical system and image sensing element.

Also, another adjustment system is available. The aforementioned errormeasurement operation of the relative position is executed at the timeof factory adjustment after the image sensing optical system 8 and imagesensing element 1 are built in the main body so as to measureinformation of drive amounts Dx and Dy of the drivers 12 and 11 fromtheir drive reference points (the drive amounts to tilt the imagesensing element 1 through tilt angles θx and θy described using FIG. 3)in correspondence with a plurality of temperatures, and measurementresults are stored in the error memory 42. Prior to the image sensingoperation, the system controller 9 reads out this information, andcorrects the relative tilt of the image sensing element 1 by controllingthe drivers 12 and 11.

The aforementioned position adjustment sequence of the image sensingelement 1 will be described below with reference to the flow chart shownin FIG. 10.

Upon detection of power ON (step S31), the system controller 9 startsposition adjustment (step S32), and reads out information from the errormemory 42 (step S33). The system controller 9 executes adjustment in theX-axis direction based on the readout information (step S34), and alsoexecutes adjustment in the Y-axis direction based on the readoutinformation (step S35). Upon completion of adjustment (step S36), theadjustment operation ends (step S37).

In this way, initial adjustment is completed, and steady image sensingcan be started.

Upon reading out information from the error memory 42 in step S33, thesystem controller 9 acquires the temperature from the temperature sensor61, and reads out information corresponding to that temperature. In thisway, a tilt angle that changes depending on the system temperature canbe corrected in the X- and Y-axis adjustment processes in steps S34 andS35.

As described above, according to the third embodiment, automaticadjustment of the relative position of the image sensing element 1 withrespect to the image sensing optical system 8 can be easily andaccurately done to fall within a predetermined range by a simplearrangement, and a further size reduction of the apparatus arrangementcan be sufficiently coped with.

When position adjustment information for each predetermined temperatureis stored in the error memory 42, changes in response to lapse of timedue to the temperature drift can be canceled.

<Fourth Embodiment>

The fourth embodiment of the present invention will be described below.

An image sensing system of the fourth embodiment initially adjusts therelative tilt between the image sensing optical system 8 and imagesensing element 1 as in the third embodiment, and the third embodimentis applied to an exchangeable lens type image sensing system.

FIG. 11 is a block diagram showing the overall arrangement of the imagesensing system according to the fourth embodiment of the presentinvention.

This image sensing system is roughly divided into a lens unit 100′ andcamera unit 200′, which respectively comprise microcomputers serving assystem controllers 16 and 18, and also error memories 17 and 19 incorrespondence with these microcomputers, and executes datacommunications between the system controllers 16 and 18. In the fourthembodiment as well, the same reference numerals denote buildingcomponents having the same functions as those shown in FIG. 7 and therespective figures of the third embodiment, and a description thereofwill be partially omitted.

In the fourth embodiment, both the lens unit 100′ and camera unit 200′have the error memories 17 and 19 for storing the results of errordetection from the reference such as the reference image sensing element40 or the like, which is done as described in the third embodiment withreference to FIG. 9. Note that the error memory 17 in the lens unit 100′stores data detected using the reference image sensing element 40 as inthe third embodiment, and the error memory 19 in the camera unit 200′stores data detected by the same method upon attaching a referenceoptical system to the camera unit 200′.

Prior to the normal image sensing operation, the system controller 16 ofthe lens unit 100′ measures the temperature in the lens using atemperature sensor 81, reads out error information (information obtainedby correcting the position adjustment information based on the measuredtemperature) from a table with a value closest to the measuredtemperature of various parameter values for a plurality of temperatures,which are pre-stored in the error memory 17, and transfers that value tothe system controller 18 in the camera unit 200′.

Parallel to this operation, the system controller 18 in the camera unit200′ measures the temperature inside the camera unit 200′ using atemperature sensor 82, reads out error information (information obtainedby correcting the position adjustment information based on the measuredtemperature) from a table with a value closest to the measuredtemperature of various parameter values for a plurality of temperatures,which are pre-stored in the error memory 19, and executes tilt controlof the image sensing element 1 by a correction method to be describedlater using the readout value and the value transferred from the lensunit 100′.

In this way, since both the camera unit 200′ and lens unit 100′ have thetemperature sensors and store error tables, in the error memories, incorrespondence with a plurality of temperatures, even when thetemperature of the camera unit 200′ is different from that of the lensunit 100′, correction can be executed more appropriately.

After that, the image sensing optical system 8 forms an object image onthe image sensing element 1, which photoelectrically converts the objectimage into an electrical signal. The electrical signal is processed bythe signal processor 3 to obtain a color video signal. The processedvideo signal is output, and mainly its luminance information is suppliedto the AE and AF units 4 and 2, which generate respectively controlsignals and input them to the system controller 18. The control datafrom the AE and AF units 4 and 2 are transferred from the systemcontroller 18 in the camera unit 200′ to the system controller 16 in thelens unit 100′ to control the iris mechanism Iris and focal pointposition adjustment optical system L4. Also, the AE unit 4 controls theaccumulation time of the image sensing element 1 for each frame asneeded.

The vibration sensor 7 such as an acceleration sensor or the like in thelens unit 100′ detects camera shake, and the shift optical systemdriving unit (AS/IS) 6 reduces blur of an object image caused by camerashake by driving the optical system L3.

Upon receiving an operation instruction signal for image sensing fieldangle (zoom) adjustment in accordance with an operator's request, thesystem controller 18 transfer it to the system controller 16 in the lensunit 100′. The system controller 16 reads out a digital cam curve fromthe internal memory (not shown) of the lens unit 100′, and controls thefocal length adjustment optical system L1, correction optical system L2,and focal point position adjustment optical system L4 in cooperationwith each other.

The correction method will be explained below with reference to an errorcorrection table shown in FIG. 12. In FIG. 12, an X-axis error indicatesan error angle and shift amount from a reference position in the X-axisdirection, and a Y-axis error indicates an error angle and shift amountfrom a reference position in the Y-axis direction.

O1−C1 and O3−C3 are computed to obtain relative error angles, and O2−C2and O4−C4 are computed to obtain relative shift amounts, thus settingcorrection data.

The initial adjustment sequence in the fourth embodiment will bedescribed below with reference to the flow chart shown in FIG. 13.

FIG. 13 shows an example of the data communication sequence between thesystem controller 16 in the lens unit 100′ and the system controller 18in the camera unit 200′.

If attachment of the lens is detected (YES in step S41), the systemcontroller 16 sends error information stored in the error memory 17 inthe lens unit 100′ to the camera unit 200′ (step S42). Subsequently, thesystem controller 18 in the camera unit 200′ receives this information(step S43). The system controller 18 reads out error information fromthe error memory 19 in the camera unit 200′ (step S44), and merges itwith the error information from the lens unit 100′ to generaterespective correction data in accordance with the aforementionedsequence. The system controller 18 executes adjustment in the X-axisdirection on the basis of the generated correction data (step S45), andalso executes adjustment in the Y-axis direction on the basis of thegenerated correction data (step S46). Upon completion of adjustment (YESin step S47), the adjustment operation ends.

In this way, initial adjustment is completed, and steady image sensingcan be started.

Upon reading out information from the error memories 17 and 19 in stepsS42 and S44, the system controllers 16 and 18 respectively acquire thetemperatures from the temperature sensors 81 and 82, and read outinformation corresponding to the acquired temperatures. In this way, atilt angle that changes depending on the system temperature can becorrected in the X- and Y-axis adjustment processes in steps S45 andS46.

As described above, according to the fourth embodiment, in addition tothe effects of the third embodiment, even in the exchangeable lens typeimage sensing system, since data associated with adjustment can beexchanged via communications between the system controller 18 of thecamera unit 200′ and the system controller 16 of the lens unit 100′, therelative state can be optimized even when image sensing optical systemshaving different characteristics are attached to the camera unit 200′.Hence, an inappropriate image sensing frame on which an out-of-focusportion locally remains (so-called local out-of-focus state) can beexcluded, and a high-quality image can be sensed.

Furthermore, since the lens and camera units respectively have memories,a highly compatible system can be built.

Furthermore, since both the camera and lens units have the temperaturesensors and store error tables, in the error memories, in correspondencewith a plurality of temperatures, even when the temperature of thecamera unit is different from that of the lens unit, correction can beexecuted more appropriately.

<Fifth Embodiment>

The fifth embodiment will be described below.

An example of the arrangement of an image sensing system according tothe fifth embodiment will be described below with reference to FIG. 14.Note that the same reference numerals in FIG. 14 denote the same partsas those in FIG. 7, and a detailed description thereof will be omitted.In the fifth embodiment, the image sensing optical system 8 and itsvarious parameters are controlled using control data of respectiveprocessors 2 to 5 (AF, AE, AS/IS, and Zoom).

The fifth embodiment will exemplify a case wherein adjustment based onespecially zoom (field angle) information in steady control ispositively done. The system controller 9 measures the temperature in theapparatus using a temperature sensor 61, reads out error informationfrom a table with a temperature closest to the measured temperature ofthose of various parameter values for a plurality of temperatures, whichare pre-stored in an error memory 42′ by a method to be described later,detects the zoom (field angle) position of the image sensing opticalsystem 8, reads out error information with a value closest to that zoom(field angle) position from that table, and executes relative positioncorrection according to a value obtained by computing the respectivevalues as needed in correspondence with a change in zoom (field angle)position.

As the correction method, the drivers 11 and 12 adjust the attachmenttilt angles of the image sensing element 1 about the X- and Y-axes. Inthis manner, the relative state between the image sensing optical system8 and image sensing element 1 can be adjusted.

As the focal length adjustment optical system (zoom lens) 11 of theimage sensing optical system 8, a zoom lens especially having a highzoom ratio is becoming the mainstream; a change in zoom position orfocal length or a change in f-number is becoming large, resulting in alarge change in focal depth. Also, high-sensitivity zoom lenses areprevalent, and the tilt direction and angle of an ideal image surfacedue to manufacturing errors often change largely.

Hence, the variation of the tilt angle of the image surface according tothe zoom position of the focal length adjustment optical system L1 ofthe image sensing optical system 8 must be corrected occasionally. Toacquire a correction value therefor, using the same arrangement as inFIG. 9 the reference imaging sensing element 40 is tilted about the X-and Y-axes at a plurality of zoom positions from the wide-angle end tothe telephoto end of the focal length adjustment optical system L1 ofthe imaging sensing optical system 8 so as to detect a tilt anglecorresponding to the maximum high frequency component, and this angle(deviation from a reference value) is stored as respective parametervalues in the form of a table for each zoom (field angle) position inthe error memory 42′. Note that the method of detecting an appropriatetilt angle can use the method described in the first embodiment withreference to FIG. 3.

This error memory 42′ comprises a nonvolatile memory such as an EEPROMor the like, and is provided to a lens unit including the image sensingoptical system 8. When this lens unit 8 is attached to a camera mainbody to assemble the image sensing system, a system microcomputer in thecamera main body can read out the angle stored in the error memory 42′.

Note that a reference image sensing optical system serving as areference may be prepared to measure an attachment position error of theimage sensing element 1 based on this reference image sensing opticalsystem, and a memory for storing error information of the image sensingelement 1 may be provided. When this memory is provided for each cameramain body in the exchangeable lens type image sensing system, aneffective system can be constructed.

In order to further improve the precision, since variations of the tiltangle of the image sensing element 1 due to a change in temperature inthe apparatus must be corrected, tables may be prepared for respectivetemperatures to store error values corresponding to a plurality oftemperatures, and can store respective parameter values. Five or sixmeasurement temperatures in 20° C. increments can be stored to haveordinary temperature of 20° C. as the center. For example, measurementtemperatures are set like −20° C., 0° C., 20° C., 40° C., and 60° C.

The adjustment sequence will be described below with reference to theflow chart in FIG. 15.

It is detected in step S51 if the power supply is turned on. Upondetection of power ON, the zoom position of the image sensing opticalsystem 8 is detected in step S52, and adjustment of the tilt angle ofthe image sensing element 1 starts in step S53. Data is read out fromthe error memory 42′ in step S54, and X-axis adjustment of the imagesensing element 1 is executed based on the readout data in step S55.Likewise, Y-axis adjustment of the image sensing element 1 is executedbased on the readout data in step S56.

It is checked in step S57 if the adjustment processes for two axes arecomplete. Upon completion of adjustment, the adjustment operation endsin step S58. After that, the zoom position of the image sensing opticalsystem 8 is monitored, and upon detection of a change in zoom position,the flow returns to step S52. An image is sensed while executing theaforementioned operation.

As described above, according to the fifth embodiment of the presentinvention, since errors of the optical system from a reference value arepre-stored in correspondence with various factors such as assemblingprecision, zoom position, temperature, and the like, even when the imagesensing optical system 8 has a tilt with respect to the image sensingelement 1 in various combinations of the image sensing optical systems 8and image sensing elements 1, good characteristics can be realized.

<Sixth Embodiment>

The sixth embodiment of the present invention will be described belowwith reference to FIG. 16.

An image sensing system of the sixth embodiment initially adjusts therelative tilt between the image sensing optical system 8 and imagesensing element 1 as in the fifth embodiment, and the fifth embodimentis applied to an exchangeable lens type image sensing system.

A video camera according to the sixth embodiment is roughly divided intoa lens unit 100″ and camera unit 200″, which respectively comprisemicrocomputers serving as system controllers 16 and 18, as shown in FIG.16, and execute data communications between these microcomputers. In thesixth embodiment as well, the same reference numerals denote buildingcomponents having the same functions as those described in FIGS. 11 and14, and a description thereof will be partially omitted.

In the sixth embodiment, both the lens unit 100″ and camera unit 200″have error memories 17′ and 19′ for storing the results of errordetection from the reference such as the reference image sensing element40 or the like, which is done as described in the third embodiment withreference to FIG. 9. Note that the error memory 17′ in the lens unit100″ stores data detected using the reference image sensing element 40as in the fifth embodiment, and the error memory 19′ in the camera unit200″ stores data detected by the same method upon attaching a referenceoptical system to the camera unit 200″.

Prior to normal image sensing operation, the system controller 16 of thelens unit 100″ measures the temperature in the lens using a temperaturesensor 81, detects the zoom (field angle) position of the image sensingoptical system 8 at the same time, reads out error informationcorresponding to the detected zoom (field angle) position using a tablewith a value closest to the measured temperature of various parametervalues for a plurality of temperatures, which are pre-stored in theerror memory 17′, and transfers that value to the system controller 18in the camera unit 200″.

Parallel to this operation, the system controller 18 in the camera unit200″ measures the temperature inside the camera unit 200″ using atemperature sensor 82, reads out error information using a table with avalue closest to the measured temperature of various parameter valuesfor a plurality of temperatures, which are pre-stored in the errormemory 19′, and executes the tilt control of the image sensing element 1by a correction method to be described later using the readoutinformation and the error information value transferred from the lensunit 100″.

Since both the camera unit 200″ and lens unit 100″ store the temperaturesensors and error memory tables in correspondence with a plurality oftemperatures, even when the temperature of the camera unit 200″ isdifferent from that of the lens unit 100″, correction can be executedmore appropriately.

An object image is formed on the image sensing element 1 via the imagesensing optical system 8, which has the focal length adjustment opticalsystem L1 for adjusting the image sensing field angle, the correctionoptical system L2 for making correction corresponding to the movement ofthe optical system L1, the shift optical system L3 for camera-shakecorrection, the iris mechanism Iris for adjusting the amount of incominglight, and the focal point position adjustment optical system L4 formaking focus adjustment. The formed image is photoelectrically convertedby the image sensing element 1, and is processed by the signal processor3 to obtain a color video signal. The processed video signal is output,and mainly its luminance information is supplied to the exposure control(AE) unit 4 and focus adjustment (AF) unit 2, which respectivelygenerate control signals and input them to the system controller 18. Thecontrol data from the AE and AF units 4 and 2 are transferred from thesystem controller 18 in the camera unit 200″ to the system controller 16in the lens unit 100″ to control the iris mechanism Iris and focal pointposition adjustment optical system L4. Also, the AE unit 4 controls theaccumulation time of the image sensing element 1 for each frame asneeded.

The vibration sensor 7 such as an acceleration sensor or the like in thelens unit 100″ detects camera shake, the shift optical system driving(AS/IS) unit 6 reduces blur of

n object image caused by camera shake by driving the optical system L3.

Upon receiving an operation instruction signal for image sensing fieldangle adjustment in accordance with a user's request, the systemcontroller 18 transfer it to the system controller 16 in the lens unit100″. The system controller 16 reads out a digital cam curve from theinternal memory (not shown) of the lens unit 100″, and controls thefocal length adjustment optical system L1 and correction optical systemL2 in cooperation with each other.

The temperature information and zoom position are always monitored. Uponreceiving a zoom position change instruction from the system controller16 in the lens unit 100″, error information corresponding to that zoomposition is read out from the error memory 17′, and that value istransferred to the system controller 18 in the camera unit 200″, thusexecuting the tilt control of the image sensing element 1 as needed.

The correction method in the sixth embodiment will be explained belowwith reference to an error correction table shown in FIG. 12. In FIG.12, an X-axis error indicates an error angle and shift amount from areference position in the X-axis direction, and a Y-axis error indicatesan error angle and shift amount from a reference position in the Y-axisdirection.

(O1−C1) and (O3−C3) are computed first. Then, correction data iscomputed by adding (O2−C2) and (O4−C4) to the computation results of{1−COS(O1−C1)} and {1−COS(O3−C3)}, respectively.

FIG. 17 is a flow chart showing the operation sequences of the systemcontroller 16 in the lens unit 100″ and the system controller 18 in thecamera unit 200″. The operation sequences of the system controller 16 inthe lens unit 100″ and the system controller 18 in the camera unit 200″will be described below with reference to FIG. 17.

It is detected in step S61 if a lens is attached. If the lens isattached, the zoom (field angle) position of the lens is detected instep S62, and error data stored in the error memory 17′ in the lens unit100″ is sent to the camera unit 200″ in step S63.

The camera unit 200″ receives the sent error data in step S64. Errordata is read out from the error memory 19′ in the camera unit 200″ instep S65, and is merged with the received lens error data in step S66 togenerate correction data, as described above.

X-axis adjustment is executed based on the generated correction data instep S67, and Y-axis adjustment is executed based on the generatedcorrection data in step S68. Upon completion of adjustment in step S69,the processing ends. After that, the zoom position is always monitored,and upon detection of a change in zoom position, the flow returns tostep S62 to restart the processing.

As described above, according to the sixth embodiment of the presentinvention, in the exchangeable lens type image sensing system, since thelens unit 100″ and camera unit 200″ respectively have the error memories17′ and 19′, deterioration of the image quality due to the tilt of theimage sensing element 1 can be corrected even when a lens is exchangedto another, thus building a highly compatible system.

Furthermore, since both the camera and lens units have the temperaturesensors 81 and 82, and store error tables in the error memories 17′ and19′ in correspondence with a plurality of temperatures, even when thetemperature of the camera unit is different from that of the lens unit,correction can be made more appropriately.

Moreover, the tilt of the image sensing element 1 can be optimallycorrected as needed in accordance with the zoom (field angle) position.

<Seventh Embodiment>

The seventh embodiment will be described below.

FIG. 18 is a block diagram showing the overall arrangement of an imagesensing system of the seventh embodiment. The same reference numerals inFIG. 18 denote the same parts as those in FIG. 1, and a descriptionthereof will be omitted.

A characteristic feature of the image sensing system of the seventhembodiment lies in that the operator inputs an instruction of anoperation mode or the like in accordance with his or her photographicpurpose, and the relative position (angle) between the image sensingoptical system 8 and image sensing element 1 is corrected to fall withina predetermined allowable range in accordance with parameter values setby the operator, thus realizing various photographic effects (e.g., thebackground is made out of focus by setting a shallow depth of field).

To realize the above effects, the operator inputs an operationinstruction (more particularly, “field depth setting value” and/or“in-focus region setting value”) to the AF unit 2 in accordance with hisor her photographic purpose prior to the image sensing operation, andthe drivers 12 and 11 adjust the tilt of the image sensing element 1 inaccordance with these values.

Reference numeral 42 denotes an error memory for storing the initialstate of the image sensing element 1, i.e., a reference state beforeimage sensing (drive amounts Dx and Dy of the drivers 12 and 11 fromtheir reference points, i.e., the drive amounts to tilt the imagesensing element 1 through tilt angles θx and θy described using FIG. 9).Based on the stored information of the initial state, the image sensingelement 1 is set at a predetermined initial position before an operationinstruction in accordance with the operator's photographic purpose isinput.

The system controller 9 controls the AF unit 2, drivers 11 and 12, anderror memory 42.

In the seventh embodiment, the attachment tilt angles of the imagesensing element 1 about the X- and Y-axes are adjusted using the drivers12 and 11 in accordance with the “field depth setting value” and“in-focus region setting value” input to the AF unit 2 after adjustmentto the initial state, thereby adjusting the relative position betweenthe image sensing element 1 and image sensing optical system 8.

Also, a readjustment system for resetting the image sensing element 1 toan initial position after the image sensing element 1 is adjusted to anarbitrary value in accordance with the operator's instruction isrequired.

Note that the setting value for setting the image sensing element 1 atan initial position is stored in the error memory 42 at the time offactory adjustment after the image sensing optical system 8 and imagesensing element 1 are assembled in the main body. In this case, driveamounts Dx and Dy of the drivers 11 and 12 from their reference points,which correspond to angles obtained by executing the error measurementoperation of the relative position between the image sensing opticalsystem 8 and reference image sensing element 40, as described above withreference to FIG. 9, are stored as initial values in the error memory42. The position of the image sensing element 1 is adjusted using thedrive amounts Dx and Dy. Since the adjusted value is stored as aninitial value in the error memory 42, the setting value used to obtain agiven photographic effect can be canceled to restore the initial valueupon completion of the photographing operation with the photographiceffect according to the operator's photographic purpose.

The position adjustment sequence of the image sensing element 1 upontaking a photo with a photographic effect according to the seventhembodiment will be described below with reference to the flow chart inFIG. 19.

Upon detection of an image sensing start instruction, it is checked if aphotographing operation with a given photographic effect is instructed(step S71). If a normal photographing operation without any photographiceffect is instructed, the processing ends since the image sensingelement 1 need not undergo any special control. On the other hand, if aphotographing operation with a given photographic effect is instructed,an “in-focus region” to be focused by the AF unit 2 is set under thecontrol of the system controller 9 (step S72), and a “non-in-focusregion” is set as the inverted region of the “in-focus region” (stepS73). Subsequently, hill-climbing control for adjusting the tilt of theimage sensing element 1 about the X-axis is executed (step S74), andhill-climbing control for adjusting the tilt of the image sensingelement 1 about the Y-axis is executed (step S75). During thephotographing operation with the photographic effect, the aforementionedtwo-axis control is always executed. Upon completion of thephotographing process with the photographic effect (YES in step S76),re-initialization is done (step S77) to reset the image sensing element1 to the tilt for a normal photographing process without anyphotographic effect.

In this way, position adjustment of the image sensing element 1 iscompleted, and normal image sensing can be restarted.

The method of setting an in-focus region and non-in-focus region will bedescribed below with reference to FIGS. 20A to 20C.

A1 in FIG. 20A indicates the entire frame to be sensed. Note that aperson on the right side of the frame is bounded by the broken line, andthis broken line region is an area which is designated by atwo-dimensional pointer called an X-Y joystick, X-Y pad, or the like andis superimposed on the image sensing viewfinder. The operator sets anin-focus region B1 shown in FIG. 20B while confirming this superimposedimage on the viewfinder.

Then, the operator sets a non-in-focus region C1 shown in FIG. 20C inthe same sequence. The AF unit 2 executes focus adjustment to maximizethe high frequency component of the set in-focus region, and controls tominimize the high frequency component of the non-in-focus region. Notethat control is made to preferentially assure in-focus of the in-focusregion B1 by setting a large weighting coefficient for an evaluationvalue of the in-focus region B1. In this connection, since a regionwhich is set as neither the in-focus region nor the non-in-focus regionis not evaluated, in-focus/non-in-focus of such region as thephotographing result is not under this control.

FIGS. 21A to 21C show another region setting method. In this example,two in-focus regions, i.e., a person and ball, are set. On the entireframe to be sensed shown in FIG. 21A, two in-focus regions B2 are set inFIG. 21B. As a result of this setting, the entire region to be sensedother than the in-focus regions B2 is set as a non-in-focus region C2,as shown in FIG. 21C.

As an example of the setting method of each region, a rectangular regioncan be set by determined two factors, i.e., the central position andsize of the region using the aforementioned X-Y pointer, or determiningtwo points, i.e., one point and its diagonal point (e.g., upper left andlower right points).

In this connection, as a special example of the X-Y pointer, theaforementioned position designation can be attained by a line of sightdetector provided near the viewfinder that the photographer looks in. Inthis case, since it is difficult to accurately designate a region, thecentral points of the in-focus and non-in-focus regions may bedesignated, and the size of each region may use a predetermined value.

In the seventh embodiment, upon focusing an object which obliquely facesthe image sensing surface, only a portion of the object is in focus bypivoting the image sensing surface in a direction opposite to the tiltand shift direction, i.e., using an inverse tilt and shift effect. Thismethod will be explained in detail below with reference to FIG. 22.

In a normal state, an object parallel to the image sensing surface is infocus, as indicated by broken lines 111 and 114 in FIG. 22, an objectwith a depth before and after the focal point is nearly in focus due tothe presence of the depth of field. Therefore, when an object is locatedon a plane that obliquely faces the image sensing surface 111, asindicated by 115, an object falling outside the depth of fielddetermined by the stop value (f-number) of a lens used (the left objectin the box 115 in FIG. 22) cannot be in focus in principle. However,when the image sensing surface is tilted to satisfy the knownScheimpflug principle, as indicated by 112, the oblique plane can be infocus independently of the depth of field.

When the image sensing surface is tilted in a direction opposite to 112by exploiting the reverse effect of this principle, as indicated by 113,an object can be set largely outside the depth of field, and ashallow-depth photo in which only a portion of an object is in focus canbe taken. Using this effect, a photo having different in-focus states onthe in-focus and non-in-focus region described in FIGS. 20A to 20C andFIGS. 21A to 21C can be taken.

In this way, by controlling/setting the angle θ to an intentionallylarge predetermined angle, a so-called “inverse tilt and shiftphotographing” technique of a silver halide camera can be implemented,and a photographic effect substantially equivalent to shallowfield-depth photographing can be obtained by focusing only an arbitraryobject.

The AF unit 2 will be described in more detail below with reference toFIG. 18.

As described above, an image signal processed by the signal processor 3is output as a video camera video signal, and is also used in focusadjustment and inverse tilt and shift adjustment in the AF unit 2.

A high frequency component extraction unit 20 extracts only highfrequency components of an image signal used to determine an in-focusstate, and an in-focus region extraction unit 21 and non-in-focus regionextraction unit 24 extract only information of the set regions on thebasis of the aforementioned in-focus/non-in-focus region information.

The information that pertains to the in-focus region is supplied to adriver 28 for the purpose of normal focal point position adjustment, andthe driver 28 controls the focal point position adjustment opticalsystem L4 of the optical system 8 on the basis of this information.

Normalization units 22 and 25 multiply by the reciprocal number of thearea ratio of the respective areas to correct the difference in area ofthe respective set regions. For example, if the area of the in-focusregion is ⅕ the entire frame, while that of the non-in-focus area is ⅘,the value of the in-focus region is multiplied by 1/1, and that of thenon-in-focus region is multiplied by ¼.

Thereafter, weighting units 23 and 26 further multiply by coefficients kand j. For a deep depth of field, the coefficient j is set to a smallvalue, and for a shallow depth of field, the coefficient j is set to alarge value.

These products from the weighting unit 23 and 26 are computed by anadder/subtractor 27, and the result is passed to the system controller9. The system controller 9 controls to drive the drivers 11 and 12 basedon this computation result by tilting the image sensing element 1 aboutthe X- and Y-axes, and detecting the tilt angle corresponding to themaximum high frequency component, thus implementing a desiredphotographic effect.

As described above, according to the image sensing system of the seventhembodiment, in an image sensing system such as a video camera or thelike in which the focal length shortens as a result of size reductionsof the image sensing element 1 and image sensing optical system 8, andthe depth of field tends to be deep, a photographic effect upon taking aphoto as if an arbitrary shallow depth of field were set can be obtainedby correcting the relative angle between the image sensing opticalsystem 8 and image sensing element 1 within a predetermined allowablerange in accordance with an operation instruction (parameter) set by theoperator.

After the photographing operation with the photographic effect,initialization can be easily done without any complicated operationsrequired to revert to a normal photographing state, and an image sensingsystem which is easy to use can be realized.

Note that the adjustment to the initial state according to the seventhembodiment can adopt any of the methods of the first, third, and fifthembodiments.

Also, the seventh embodiment can be applied to an exchangeable lens typeimage sensing system described in the fourth and sixth embodiments.

In the first to seventh embodiments, the relative position between theimage sensing optical system 8 and image sensing element 1 is adjustedby changing the tilt of the image sensing element 1. However, thepresent invention is not limited to such specific method. For example,an optical member for independently adjusting optical path lengths inthe X- and Y-axis directions may be inserted before the image sensingelement, and the relationship between the optical member and the imagesensing element may be optimized by adjusting the optical member aboutthe two axes.

It should be noted that the present invention includes combinations ofthe aforesaid embodiments or technical elements disclosed therein.

Further, the present invention also includes an apparatus, formed by allor a part of the apparatuses disclosed in the embodiments of the presentinvention, which can be used either individually, with anotherapparatus, or within another apparatus.

Further, the control method of the image sensing system of the presentinvention can also be achieved by providing a storage medium storingprogram codes for performing the aforesaid processes to a computersystem or apparatus (e.g., a personal computer), reading the programcodes, by a CPU or MPU of the computer system or apparatus, from thestorage medium, then executing the program.

In this case, the program codes read from the storage medium realize thefunctions according to the embodiments, and the storage medium storingthe program codes constitutes the invention.

Further, the storage medium, such as a floppy disk, a hard disk, anoptical disk, a magneto-optical disk, CD-ROM, CD-R, a magnetic tape, anon-volatile type memory card, and ROM can be used for providing theprogram codes.

Furthermore, besides aforesaid functions according to the aboveembodiments are realized by executing the program codes which are readby a computer, the present invention includes a case where an OS(operating system) or the like working on the computer performs a partor entire processes in accordance with designations of the program codesand realizes functions according to the above embodiments.

Furthermore, the present invention also includes a case where, after theprogram codes read from the storage medium are written in a functionexpansion card which is inserted into the computer or in a memoryprovided in a function expansion unit which is connected to thecomputer, CPU or the like contained in the function expansion card orunit performs a part or entire process in accordance with designationsof the program codes and realizes functions of the above embodiments.

In a case where the present invention is applied to the aforesaidstorage medium, the storage medium stores program codes corresponding tothe flowcharts in FIGS. 4, 6, 10, 13, 15, 17, and 19 described in theembodiments.

The present invention is not limited to the above embodiments andvarious changes and modifications can be made within the spirit andscope of the present invention. Therefore to apprise the public of thescope of the present invention, the following claims are made.

1. A method of controlling an image sensing system which comprises animage sensing element for photoelectrically converting incoming lightfrom an image sensing optical system, wherein information that pertainsto a relative position of the image sensing element with respect to theimage sensing optical system is stored, and a position adjuster iscontrolled to adjust the relative position of the image sensing elementwith respect to the image sensing optical system on the basis of theinformation, wherein the image sensing system has a first memory thatstores an adjustment amount used to correct a tilt of an optical axis ofthe image sensing optical system with respect to a reference plane, anda second memory that stores an adjustment amount used to correct a tiltof the image sensing element with respect to a reference line, and uponthe adjustment, the adjustment amounts are read out from the first andsecond memories, the readout adjustment amounts are merged, andadjustment is done using the merged adjustment amount.